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ABSTRACT . ' 

Several factors thought to influence the 
representativeness of behavioral assessnent d^ita were examined in an 
analogue study employing a nultif actorial design, systematic and 
unsystejiatic methods of observing group behavior were investigated 
using tji male and 18 female observers.^ Additionally, valence 
properties of the observed behaviors njferfe inspected. Subjects' 
assessments of a videotape were compared to a criterion code which 
defined the populatibn of behaviors. Results indicated that 
systemaTic observation' procedures were more accurate than 
unsystematic procedures, though this factor interacted with gender of 
observer and valence of behavior. Additionally, males tended to 
sample, more representatively than females. A third finding^ indicated 
that t^^e negativ^ely Waleuced behavior was over6stimate.d, while the 
neutral and posi|:ively valenced behaviors were accurately assessed. 
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Abstract 

Several factors thought to influence the representativeness of 

' behavioral assessment data were examined in an analogue study 

employing a multifactorial design. Systematic and unsystematic 

methods of observing group behavior were investigated using 18 male 

*and 18 female observers^ Addition^ally , valenCe properties of the 

observed behaviors were inspected. Subjects' assessment© of a videjb- 

tape were compared to a criterion code which jde^fined the population 
• 'I* > . j 

of behaviors. Results indicated that systematic observation procedures 

were more accurate than Unsystematic procedures, though this factor 

Interacted with gender &f. observer and valence of behavior. Addi- 

tionally, males tended to sample more representatively tt}^ females. 

A third finding indicated that the iregatively valenced behavior was 

overestimated, while the neutral an3 positively valenced behaviors 

were accurately assessed. 
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t)bservation Procedure, Observer Gender, and Behavior Val^^^ as 
Determinants of Sampling Errors in a Behavior Assessment Analogue 



The enterprise ^ behavi\Dr fherapy relies largely on the 
direct observation of behavior, thus the improvement of tMs 
''methodology is perhaps the most critical task for strengthens^ 
our contribution to the sciehce of ivuman behaviior" (Johnson & 
Bolstad, 1973, p. 8). Underscar/ing this need, Linehan (Note 1) 
has called for an examination of the validity of behavioral assess- 
ment techniques* . ^ • 

i .. , ... ■ > 

Jones, Reid, and Patterson (1975) have suggested that the 

definitive features, of naturalistic assessment are (a) the sampling 

of behavior in situ, Cb) the low degree of inference in describing 

overt behavior, and (c) the use of trained and impartial observers. 

However, the impartiality of observers and, of the data they record 

has been reqently qu^tioned along three lines: (a) social 

influence processes, (b) the adequacy of sampling of the population 

of behaviors, and (c) the gender of observers. In the analogue 

study to be reported here, these three factors are analyzed, both 

for their Separate effects and their interaction^ . 

Social influence factors have been studied by Scott, Burton, 

and Yarrow (1967) who found that an observer informed of the (nature 
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of the research collected data that- confirmed the hypothesis sig- 
nificantly more than. did uninformed observers. Because of the 
importance of this finding to behavioral scientists and because 
. Scott was the informed observer, additional research has been 
conducted on this bias phenomenon.. There is conflicting data as 
to the effect of social factors on biasing observations (e^g., 
Kent, O'Leary, Dipment , & Dietz, 1974'; Shulle^ A McNamara, 1976; 
Skindrud, 1973). However, the. preponderance of the evidence 
aiggest that under certain conditio?^ both behavioral' and global 
perception data are subject tp biasing influence (see also O'Leary, 
Kent, & Kanowitz, 1975). ^ — ^ ' 

.An additional variable which may affect the data 'collect?ion 
process Is the definition of the behavioral code, and the presumed 
relationship between the code and the personality construct which ' 
it is designed to assess. For example, behaviors -such a? "high- 
rate"; or "sel^-stimulation'V (Jones et al., 1975) ate implicitly 
or explicitly related to constructs such^as ''hyperactivity/' and 
thus are negatively valenced. Whether observers differentially 
perceive and record behavior as a funcffo^of valence prnn- ■ 
af code-construct relationships is critical, especially . p 
most )ding systems 'contain behaviors of a range of valences, 
/i second factor which has been shown to influence the 
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representativeness of behavioral data is the procedure df sampling 
employed. Thomson, Holmberg, and Baer (1974) have shown that a 

spaced sequential procedure of observing groups produces the 

if . ' - * ■ 

Smallest error in estimating the population of behaviors. Powell, 
Martindale, and Kulp (1975) have demonstrated |>hat momentary time 
sampling at least once every 120 sec provides data which are truly 
representative of the population*. In general, th^| mdre frequent 
the sampling of the population, the more representative were thie 
data obtairted. In a co^arison of time-sampling and interval- 
sampling procedures/ Repp, Roberts, Slack, .Rep^p,. and Berk3,er (1976) 
found that the most representative dat^ were obtained in interval 
recording, while time-sampling procedures ''produced highly inaccu- 
rate estimates of the population of behaviors. - However, both of 
,these latter studies irtclude a numbier of deficiencies trti^h ' 
limit generality. Tot exapple, Ppwell et al.^used a simple one- 
categ^iry^ code of a Secretary's in-seat Jbehavio^., wliile Repp et al. 
employed a time-sampling spacing which yielded infrequent samples 
of behavior- In addition, because Repp et ;al.. used eleCtrb^ 
mechanically-generated charL records as the population frtJm which 
samples were derived, the. artificial nature of this study pre- 
eludes generalization to research employing human observers: 
They must decide on the par ticular person to be sampled ady w^ll as 



- ^ . . Determinants of Sampling Errqr 

the ^ongoing behavior , ' and must ensure atciiracy of data logging. ' 

Kubany and -Sloggett (1973) provided .an excellen^comparison 
of the representativeness of two population-sa^lpling^ iprocedur6s . 
In their study, dat*a collected by a teacher using a VI-4 min * 
schedule of cla^si^'feom observation were compared to data colljected 
by the experimenter on a FI-15 sec schedule. Kxibany and Sloggett' 
results indicated that momentary time sampling at logger Intervals 
provides data which ar6 'veridical with more frequent sampling of 
the population. A limitation of this study, hofe^, is that the 
t/ue population, of be^iaviors was not assessed. - Thus, there may 
have been constant error in either over-^ or underestimation of 
the sampled behavior's. 

The nature ol the sampling .procedure is particularly impor- 
tant when the behavior *of groups i^' the focus of stu(^y ''^.g., 
classrogms).'* It has long^een suspected that .teachers are unsys- 
tematic in their sampling procedures, ^'hd a random manner of obser- 
vation may Je^^d to inaccuraf^ies in sum^iary 'reports (Wahl^ & 
LesTce, 19/3) and behavioral data. However, there has been no 
empirical test of whether systematic methods of sampling improve 
upon those data collected from unsystematic recording procedures. 
,Th^s question is especially salient because of the recent emphasis 
on the ^collection of normative data (e.g.; Walker & Hops, 1976). - 
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In addition, one migfit expect that the valence properties -of *the 
behavioral codes employed may Interact In somjs^ fashion with the 
observation system used. ^ ' * 



A third variable., which has not received attention In the 

behavioral .assessThent literature. Is ;he- Influence of gender on 

accuracy of observation, J^Dafa from the literature on social 

memory suggest that females make more accurate assessments than 

tnal^s after observing videotapes or photographs ef groups pf 

, * ' _ - . / 

people (Mazanec^S McCall, 1975; Wltryol & Kaess, 1957), However, 

these assessments are not made Immediately , and, thus dlffibr from. 

most behavioral assessment tasks. The vigilance literature 

suggests that females may. be more accurate l4i tasks -requiring 

sustained attention (Davles & Tune, 1969), althou^ males iave 

been found to have a shorteT: reaction time in visual, vigilance 

tapks (Harkins, NowlJ.n, Ramm, & Schrbeder, I97*A; Tcrlin ^ Fisher, 

r . ■ ^ 

1974) and to be superitrr in problems which recf'uire visual'* 

. ' I 

discrimination of objects set in a lapger context (Guilford.- 1967; 
Macc^oby £t'jacklln, 197Aa; 197Ab). The inconsistency and lack of 
difect generalizabi 1 1^ of tii^ present data pase necessitates' ^ 
examination of possible gendol dif f erences ^ in th^ edacity to 

- ' 

representatively sample populatift>ns of behavior in a ^naturalist ic 
assessment task. . " 
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' . ;■' '• ■ ' ■ '■ ■ 

. The preisent experiment was designed to Investigate three 

^factor's which possibly inffuence the representativeness of data 

obtained in studies employing b'^t^vioral assessment '^n ge'aefal, and 

'momentary time samplin'g in particular, , This study is not proposed 

, as ..representative of traditional behavioral data' collection; for 

example, we employed videotapes; 'father, tlian live p^rf orders, in 

order to assess the actual population of bertaviqrsr Rather, our 

purpose is to contribute to knowledge pf those' factors \&hich^ may 

'bias observation, through an analogue stucjy x^ith high' internal ' 

validity, and which allQws for higher-order interactions to be 

revealed. First, bias fefffects due t:J' valence properties of code- 

construct relationships were inve;3tigated using behavioral codes 

to represent negative, -jieutral,** ^nd positively T^lenced personality 

characteristics.. Second, the proces's of sampling groups was 

examined, atid an unsystematic (idiosyncratiJ^) method of abservation 

^ ' . - ' , ' - ^ -f^ 

was compared with a systematic (standard)' pr^ceduTre. Third, the 

•[ ■ . ' ' . / ' - m 

represeatotiven'&Rs of sampling a^i a function of observers' gender 
was investigated. Additionally, both algebraic and absolute 
d«^viation (Newell, 1976) in rsamplitn^ th§! true populf^tiori of behaviors 
were assessed. Finally, stepwise regression was used tb^examine 
predittors of absolute errov . . v _ ' 
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Metho d 

t . ■ — 

Subjects - f ^ 



^ Flfty-three\subjects between, tbe ages of-18,,and 42 years 

<inean=22.6) participated in this study as obseryers. Eighte^ ' 

\ ■ - 

male and 18 female observers are included in- the 'final safuple; 

eiftht males and pAne females were ^excfluded on the ^asls of the 
^election criteria whichfare detariled in the Design and Procedure 

Section. Subjects had an average of 2.1' years of college 

education, with a mean of 3.8 psychology courses. All sbbjects^ 

received coarse credit for their participation. 
' Matet'ials v ' . ' 

In order to specify the population of behaviors contained 

within the assessment stimuli, a videotape was produced which 

depicted six people (two male and f our^f eraale) who were ^instructed 

i . * I ■ 

to follow an informal script. Jhis group tape was 10 min long; 

and designed to resemble the' admlnistrati?on of a group personality 

. • ■ " - ■ t ' 

questionnaire. The actors pat about a large conference ^ble, and 

' . X 

were instructed to behave as if they wejrd talj,ing a* test", although 
they were- Lpld to engage in tjie follo^g behaviors of interest: 
(a) holding the t^st booklet* in, both> hands ; fb) adj\isting clothing, 
jewelry, and so for th,;' resting- head in bott^ hands; and (c) cross- 
Ijig.the arms. TheseV:apecif Ic behaviors defined the population S^f 
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Behaviors whHcH\subjects were to later 5^ampie, ^ 

' ' '^^ \ ^ ' \ ' ' ' ^ 

After the videotaping wascompleted, a signal whic^ noted 

'10 sec inter\>ads was dubbed onto the audio track." A criterion ^ 

code of all actors' beha\Aors at all intervals (define'd as the 

instant the signal sounded) was^ oopstfucted^ by the author a 

female assistant. Disagreements were resolved in consultation 

wtth other assistants' who were either male or female; The group 

I ^* ^ / 

tape' that was employed hfed relatively equal percentage^ of 

intervals .during which The^ actors engaged in the three valenced 

behaviors (Neutral=10 . 6^; Negative=8 , 3% ; and Posftive=13 . 1%) . ' 

Because 68% of the^^corable intervals contaiixed- nonsalieht 

behaviors, over- or undersair.pling of one of the behaviors of. 

interest did not necessarily determine ^he sampling probability 

of another recordabl'e beh^viof. All tapes were presented o4 a 

'black and white video monitor (Sony model CVM-llOUA), ^1 su|)jects 

-a ' 

recorded -^eir observations on a three-item checklist. 
De&i.fin «ip4_P^roredur€=^ 

T^, ' ' -ft • ' ' ^ 

This study used a three-way ^d^igu, with two levels of each 
between groups factor (observation procedure and observer gender)r 
and three l^v^ls of the within subjects factor (code valence). ' 

The investigation involved a single laboratory session, with 

/ > . ■ ' 

subjects randomly assigned to observation procedure groups. All 

if 
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\ . ^ . - . 

levels of the within subjects factor wore obtained fr9ni the single 

■■ ■ i)-^ ' 

' checklist , completed l>y each obs(^rver. • 

• ■. ^ ^ ^ ..." , ' ' 

Subjects 'were recruited from two psycho£ogy lecture clag'ses,"'' 

' -/'^ " ■■ " * • - . ■ 

with the exp§rlnienrer ireques.tlng "research assistants" to aid in 

- -i" . C ' ■* . 

a behavioral ohservaplon project. Upon arriving at tjie laboratory, 

^ : . V ' •• • , ■ ' ■ ^ . • J 

^subjfe^tjB completed an "appa^icatlo'n for assistantshlp" which 
-rtquested basic demographif Information. After answering / 
preliminary questions, the first instructional tape was presented 
to the subjects, who were seen in\groups numbering up to five. 

Experimental deception , A deceptive set was Introduced -to 
provide for valenced . behavioral codes . , Ostensibly , the observeYs 
were to code ovei^t behaviors of people talcing a gt^oup personality 
test. The study was, supposedly, an attempt to compare two 
methods of personality as^sessment, and the observer s" were toM 
that thair observations were to be lat.er correlated with the^^' 
questionnaire measures. The observers were needed to "ensure 
that experimental bias would be eliminated, as the experimenter 
had already scored th^ personality tests/' 

Further instructions were delivered on videotape, to insure 

standardized presentation of information. The behavioral codes 

/ 

were defined and modeled, and a rationale was offered for the "* 
scoring of the particular b'^haviors. It was explained that>an 
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earlier study had .found relationshfip^i hetv^en these pvert 
behaviors and the personality constructs which were measured In ' 
the test. . "Reuroticism," the negatively valenced behavior , 
ostensibly was- related to a|category called "Fidget" — the manlpu- 
latlon of clothing or, jewelry, or holding the ho^d In' both hands. 
"PoBltlve Functioning," the posftlvely valenced behavior, was said 
to be related to "Bookholdlng," or holding the testis booklet with 
both hands. Finally, subjects were told that a "neutral" behavior 
needed to be observed, that being the "Armcrosstng" category which 
was supposedly related to no particular personality construct. 

Training tap es. To ensure that all subjects could 
accurately assess the behaviors of intercut ,'^^three videotapes of 
irrrf'lvidual females were used for training purposes. During each 
of t;heso 5 min t.ipes, questions rould be asked, and definitions 
of the boliaviors were again offered. ^ A fourth videotape was 
employed, to assess accuracy of code usage, with an^ interval-by- 
interval agreement score (Hawkins & Dot son, 1975) of 90% 
required in order to view the group tape. 

^ Pbservat lon _prxijredj^ijrG^_j^ tape . After the subjects * 

accuracy was assessed, a se^iond instructional videotape was 
presented which explained tlie coding procedure to be us^d in 
assessing a gronp^ of indivi^luals . Half of the observers were 



Determinants of Sampling Error 

taught to^observe the groqp^ systematically. (**Start with, any 
individual , t then observe each person for ,one interval in a, clock- 
wise fashion until the tape ei>d's.") The remaining observers^ were 

asked to use c-xn unsystematic observation strategy. (''Start with any 

.1 

individual, choose the next person to code- in a completely random . 
Tanner; do not use any system which would allow prediction of the 

>persdT^- you will choose.") All subjects were then cautioned 
that they tnust Observe the individuals in ttie group in ^systematic/ 
unsystematic fashion "to ensure th^t the data will be free from 
blas.'V 

Immediately before observing the group tap^,"all subjects 
were asked to define the behaviors of interest', and to state the 
observation procedure to be employed. 6ach subject observed the 
group tape in isolation; all were told that they were viewing a 
unique tape. 

After the group tape was completed, a questionnaire was 
given to check the adequacy of the ^deceptive set. Subjects were 
asked first to state the relationships between the behaviors and 
the personality constructs^ and second, to state the degree of 
their beliefs in these relationships. Only those subjects who 
stated at least two of the relationships and indicated at legist a 
mild belief in thom were included in the present analysis. 
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, / - Hesul t s . 

•Of initiar^j^terest .was the assessment of possible 
differences between subjects -Ci^tained or excluded -from the primary 
analyses on the basis of rejection criteria. The t tests performed 
indicated no group differences with respect to age,, class standing, 
number of psychology courses taken, grade-point 'average C^tPA) , or 
pretest accuracy. The largest (though still nonsigrjif icant) 
difference was found on the absolute error variable, with -subjects 
included in the analysis making an average absolute error of. 10.7% 
while Excluded subjects deviated 8.9%, _t(51)«-l .66, £>.10. Devia-y 
tion for this aod all other analyses was defined as the observers' 
estimation of the percentage of intervals in which the group • 
engaged in specific behavior irflnus the true percentage of intervals 
the j;roup engaged Ln these behaviors. In addition/ If compared to 
the actual population of behaviors, a relative indication of over- 
And underestimation is available. ^.^ 

Prior to analyses of the primary dependent Measures, a two- 
way analysis of variance (ANOVA) was used to determine whether each 
^observation/gender group retained in the analyses were equated on 
demographic, measures of deception, ' and accuracy of pretest 
variables. There were no significant group differences on age', 
class standing, CPA, pretest a'^curacy, number of relationships 
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remembered, or the decree of belief In these relationships. For 
all subjects, the average belief rating of the negatively valenced 
category was with ^Jfhe positively valenced category rated 

2.77 (l=not -related^ 5=very strongly related), _t(70) = .87, £>.10. 
However, one ANOVA did' arevfeal a significant source, with females 
having taken more psychology courses than males, F( 1 , 32 )=5 . 66, 
£<.025, 

■ When ana ly2?S^hg deviation scores, the dependent measure may be 
conceptualized as either the long-range average of observers' 
error (algebraic error) or the average error independent of 
direction (absolute error), ^'in the present experiment all 
•analyses ompyloying algebraic error were analyzed via three-way 
ANOVA. Absolute error was cf^nsidered primarily relevant to the ^ 
between group factors, ;^nd was analyzed xislng a two-way ANOyA. 
^ Analysis of algebraic error indicated several signif idlj>t^ 
sources of variance. Systematic observers were more* accurate than 
unsystematic observers, with the former making deviation errors of 
-.23%, an^ the latter making errors' of 2,. 57%, F( 1 , 32) = 16 . 09 , 
£C;.001. Thus, in terms of* total recordable behaviors, unsys- 
t^^matic observer sjperceive'i 3.7% more behaviors than were actually 
present! while systematic observers actually underestimated the 
lotal behavior by ,3%. The observation procedure, however. 
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itvterac ted .with the,, ob^ierver s • gender , • R( 1 , 32)=4 , 28 , £*^.05 
(s^e Figure 1), though it^id mot interact " v/it'h code valence,- ^ 
Category valence was a third source of significance, F(2,64)= 
1^3.92,. .001, with an a posterior i test ^ indicating that more 
deviation errors were ma.de in observing the n(?gatively valenced 
^ category (3.52%) tha^ri either the positively (-.18%) or neutral^ly 
(..47%) ya?.enced categories, _t(64)=4.98, £<.001. ly terms of the 



papula tion^ of behaviors, observers overestimate'd thg: negatively 
valenced and neutral category by 42.4% and 1.6%, respectively, 
Vhile. underestimat iog the positive category by 1.3%. Addition- 
^lly, a valence x observation procedure x gender interaction was 
sl^^^f icant ^ F(2,64) = 5.03^ £'<.01 (see Figure 2)/ 




Figure 2 approximately here 

two-way ANOVA for absolute er^or indicated that males 
were, mow nccur:\tc in sampling the population tlinn fomalGS,' 
£(1 , 32)=(>.95, £<\0i5. Malos dovintcui an average of/9.06% while 
females deviated an .average of 12.28%. There were tio additional 
sources of variance for absolute error. 

Stepwise regression ana lyses were performed on both absolute 
error, as well as the number of errors made on the pretest 



i 




)• 



... I • V • 

Determinants of Sa^npllng Error 

' ^ ' ' . 16 \ 

(accuracy of using the codes).' After controlling for the sources 

i of error. which constituted the composite absolute error score, no 

demographic variables predicted absolute error. However, GPA was a 

significant predic^tor of errors on ^the pretest-, with a partial 

correlation of -.39, F ( 1 , 34) =6 . 17 , £ ^ . 05 . Thus, the lower the 

GPA,^the more errors were made on the test tape. Of additional 

importance, males tended to have lower grade. point averages ^ ' y 

although this was not a significant source lof variance (£>.10). 

^Iscfus&ion \ 

♦ 

. It appears that systematic momentary time -sampling procedures 
provide relatively unbiased information, hile unsystematic time 
sampling produces a significantly larger degree of population- 
sampling error. This result extends the work of Powell et al. 
(1975), a-nd partially confirm? speculation by Oma^, Fiedler, 
* and Marvin (19^6) that unsystematic methods of observation allow 
idiosyncratic bias. The degree this idiosyncratic bias is 
evidenced by a significant gender x valence x observation 
procedure interaction, and insp^cti^o'^ Figure 2 suggests that 
sampling error is actually naultidetermined . > However, no inter- 
prctation of this interaction will be* attempted . We would like to 
note, though, chat behavioral assessment, 1^%^ other subdiscipl ines 
/of psychology ,*^appears to be subject to higher-order interactions 
(cf., Cronbach, 1957; 1975) uid continued investigation of 



O 7 Q 

ERIC ^ 



Determinants of.,. Sampling Error 

- ■ 17 

attributes and treatments which may effect observer ^behavior 
appears to be in order. ^ C * 

An unexpected finding was that females; tended observe less^ 
representatively than males when absolute error is the measure of 
int^r^st. ' When algebraic error is inspected, females appear to be 
less, representative observers when using a random method of 
observation. The lack of main effect for observer gender in the 
algebraic etror Analysis ^fleets the dispersion of femalie* errcJr . 
in^lipth the positive and negative direcf 

prf a level of error approaching zero. iCionale for these 

sex differences may be found in the literature on visual discrim- 
ination (Maccoby &' Jacklin, 1974a; 1974b). However, the present 
findings should be considered only as suggestive for further 
research, and certainly not as an adequate basis for personnel 
decisions. 

An additional note of importance i^ the failure to detect 
differences in the rate of absolute error between subjects included 
or excluded from the. primary analyses. While this suggests a degree 
of j^ener.ility of our findings, a more sensitive test would have 
required a larger sample of subjects who did not believe in the 
deceptive code-construct relationships. 

In conclusion > valence of behavioral codes, gender of <\ 
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observer^, and the observation procedure have been demonstrated 
to influence the representativeness of momentary time sampling In 
a behavioral assessment analogue. It is clear that If behavior 
t^iel-aplsts are to continue to rely on their data as evidence of 
the efficacy of their techniques, rather than on testlmonlaJ.s 
(Johnson & Bolstad, 1973), they must eliminate or com 
ynoijAraridom sources of populat.o.i-sampling e^or. First, however, 
the complexities inherent in behavioral observation must be 
thoroughly studied. j ' • * 

It is likely that the effects of s^oclal influence and j 
gender can be negated by careful observer training, just as the 
adoption of systematic sampling techniques has produced more valid 
estimates of time frequencies. The value of analogue studies lies 
ln> identification of those- biasing factors to which trainees 
should attend. 



Determinants oJ^ Sampling Error 

■19 



Reference Notes- 



Linehan, M. }/(,/ Content validity" In behavioral assessment . 
Papey presented a* 84th^*»nnual meeting of. the American 
Psychological Association, Washington,, D. C, , September 
3, 1976. ■ . 



V " Peterijjltlant? of Sampling Error 

^ i . , . ■ ' . ^ ' 20 

■ • ' ; . ■ - / 

. , ' Reference s ^'-^ ^' 
Cronbach, L. Js, Beyond the two disciplines of scientific . 
psychology. Aqiericaq' Fsycho'logist , 1975 , 30, 116-127. 

'1 . • : . ' 

Cronb^ch, ,i. J. The two ^disciplines of seieT^-tific psychology. 

, American Psychologist , 1957 , J^, .671-684 . ' ^ . ' 
Davies, I>; R., & Tune> G. S. Human vigilance performance . 

New York: American Elsevier, 1969 • / 
Guflfo ^d, J^. P. The nature of human intelligence , ^ew York: \ 

McGraw-Hill, 1967. , . 

Harkins, S. W., Nowlin, J. B., Ramm, D., & Scjhroeder, S*. , 
'Effects of age, sex, and time-on-watch on a brilaf A 
continuous perforjttance task. -In E. Palmo^e (Ed.), Noi:m^l ^ 
\ Aging (Vol. II). Durham," North Carolina:' Duke University 

) 

f Press, 197A. 

■ ■ * 
Hawkins, ^R. P., & Dotsbn, V, a'. Reliability ^scores that delude: 

( An Al-ice in Wonderland trip through the misleading charac- 
teristics of inter-observer agreement scores in interval 
recording. In E. Ramp & G. Semb (Eds.), Behavior analysis : 
Areas of research and application . Englewood Cliffs, N. J.: 
Prentlce-Hall, 1975. 

* 

Johnson, S. M. , & Bolstad, 0. D. Methodological issues in 

naturalistic observation: Some problems and solutions for 
field research. In L. A. Hamerlynck^; t. C. Handy, & ' 
£• J. Mash (Eds.), Behavior change: ^Methodology, concepts 
and prnctlcey .. Champaign, Illinois: Research ]^ess,^^^73. 



Determinants' of Sampling Error 



Jwics, K., : ^ Pnrtcrson, C; , Natural l.stJc 

observation, in ,^tfnll^al assessmoni: . Tn P. McReynolds (Ed-.), 
♦ Advances in prsychological assessment (Vol . lit). Fran- 

Cisco: Jossey-«Bass, 1975. ^ 
Kent, R. N., O'Leary^;^ KV D. , Diament , C. ^ & Dietz, A. Expectation 

biases in observational evaluation of therapeutic change. 

Journal of Consulting and' Clinical Psychology , 1974, 42^, 

774-780. • 
Kub^ny<, E. S.;, & Sloggett, B. B. Coding procedure for teachers* 

Journal of Applied/ Behavior Analysis , 1973^ 6^, 339-344. 
Maccoby^ E. E., & Jacklin, C. N. The psycho l<^gy qf sex diffe^p - 

ences. Stanford; Stanford University Press, 1974, (a) * 
Maccoby, E. E., & Jacklin, C. N. Myth, reality, and shades of . 

gray: W^t ; we know and don't know abouWsex differences. 

Psychology Today , 1974, 8(7), 109-112. (b) 
Mazanec, N., & McCall, .G. J. Sex, cognitive categories, and 
< observational accuracy. Psychological Report^ , 1975, 37, 

987-990. * 
Newell, K.« M. More on absolute error, etc. Journal of Motor 

9 - — '■— - - — 5 . 

^ Behavior , 1976, 8^, 139-142. ' 

0'I.eary, K. D., Kent, R. N., & Kanowitz, J. Shaping data col- 
lection congruent with experimental hypotheses. Journal i' 
of Applied Behavior Amilysis , 1975, 8, 43-51. 

Omark; D. • Fiedler, M.\..» &*Marvin, R. S. Dominance hier- 
archies: Observational techhiques applied to the study of 



23 



1. f 



DeteTrni,nants. of Sampling Ertor 



^ r " children at" play a Instruct ianal Science ,^ 1976, 2» -^03-4^^ 

Powell, J,, Martindple, A.,' & Kiilp, S, An evaluation of time-^ 

^ \ sampling ^measures .of behavior. Journal of Applied 

. , BehavioVal Analysis , 1975, 463'-469, V *; 

^ . ■ ' ' . . .* 

• , Repp, A. C, Roberts, D. M. , Slack, D. J,, Reipp, C, F,. 

' Berkler, M. S. A comparison of frequency, interval, and 

time-sampling methods of data collection. Journal of 

Applied Behavior Analysis , 1976, 9^, 501-508- 

' Scott, P. M., Burton, R. V., & Yarrow, M, Ri Social reinforce- " 

ment under natural conditions. Child Development , 1967, 38 , 

53-63. . ' ■ . 

Shuller,^D., & McNamara, J. R.. Expectancy factors in behavioral 

observation. Behavior Therapy , 1976, 7» 519-527, / 

Skindrud, K, Field evaluation o/* observer bias under ove¥t. and ^ 

covert monitoring. In L, A- Hamerlynck, L. C, Handy, 

.... ^ • . / 

^ 

E, J, Mash (Eds,), Behavior change; Methodology, concept s,. 
and practice , ^ Champaign, Illinois: Research Press, 1973.^4 

Thomson, C,^ Holmberg, M,, (& Baer, D, M, A brief rep^t on-a^^ 
' ' comparison of time-sampling procedures, Jburnal of Applieid 
Behavj.or Analysis , 1974, 7, ^23-626, ^ 

Tplin, P., & Fisher, P, G,- Sex differences and effects of irre- 
levant auditory stimulation on performante of a visual 
vigilance task. Perceptual anc} Motor Skills ' 1974, 2i» 
1255-1262. ^ » . • 

Watiler, R. C., & Leske, G. Accurate and inaccurate observer 



^ ' Determinants of Sampling Error 

■ ' * ' • , * 

summary reports. Journal of tJervous and >lental Disease ; 
- 1973, r56, 386-39A. 
, Walker, & Hops, H. Use of normative pe^r data^as a 

standard for evaluating classtoom treatment effects. 
Journal of Apj)lied Behavior Analysis , 1976, 159-168. 
\ Witryol, S. L., & Kaes-s, W. A. Sex differences in social memory 
tasks. Journal vc^f Abnormal and Social Psychology , 1957, 
54, 3^3-346. . * 



( 



Determinants of Sampling Error 



Footnotes 



This study was truly the result of the effort 'of a number 

■ ■ ' ■ • •..■;/■'■ 

of individtmls , and the author is iiu?ebted to/the entire staff ^of 
the Kamehameha Early Education Prograiu for facilitating its out- 
, come. In particular, Larry Loganbill was an outstanding^ audio- 
vislial resource, Lou Ann Wooddell showed dedicated acti^i^g and 
coding ability, and Mary Hinck deserves plaudits for ber consis- 
tent and good-humored help. Kim C. M. Sloat was instrumental in 
all phases of this project, and his support is gratefully ' 
acknowledged. 

This paper ic part of a thesis subinitted by the first 
author to the graduate division of the University -of Hawaii in 
partial fulfillment of the requirements for the M.A, degree in 
psy c ho 1 

Requests for reprints of this article may be addressed to 
Gary M. Farkas, Department *of Psychology, University of Hawaii, 
2A30 Campus Road, Honolulu, HT , 96822. . 



) Deterrolnahts of Sampling Error 

25 

Figure Captions ■ ^ 

Figure 1, Sampling •'Error as a Function of Gender and- 

Observation Procedure. ^ *: 

Figure j2./ fiampling Error as a Function of Ge.nder; 

Observation ^oci^Sre*, and Valence. 
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